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Abstract 

We show that there are infinitely many binary strings z, such 
that the sum of the on-line decision complexity of predicting the 
even bits of z given the previous uneven bits, and the decision 
complexity of predicting the uneven bits given the previous event 
bits, exceeds the Kolmogorov complexity of z by a linear term in 
the length of z. 
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1 Introduction 

On-line decision complexity has been introduced and investigated in [?, 
?]. It also naturally appears in the definition of ideal influence tests [?, ?]. 
A natural question is whether algorithmic mutual information of two time 
series x, ?/, can be decomposed into an information flow going from x to 
y, a flow going from y to x, and an information flow instantaneously 
present in both strings. It turns out [?] that this question is related to 
the question of defining a decomposition of K{x,y) with l{x) = l{y) as 
the sum of the complexity of predicting Xj+i given xi...Xi and yi...yi, 
i ^ n, and the complexity of predicting ^j+i given xi...Xi+i and yi...yi. It 
will be shown that using on-line decision complexity for this complexity, 
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this sum exceeds K{x,y) by a linear constant in l{x). A modification of 
tliis definition of on-line decision complexity will be shown to have an 
approximate decomposition [?, ?]. 

Non-additivity of decision complexity was also shown in [?], in the 
context of randomness defined by supermartingales. Using natural def- 
initions for randomness a paradox is shown: if the even bits of z given 
the past uneven bits of z arc random, and also the uneven bits of z given 
the past even bits of z are random, than it is possible that z is not ran- 
dom. The proof of this result implies that additivity of on-line decision 
complexity is violated by a logarithmic term. 

2 Definitions and notation 

For excellent introductions to Kolmogorov complexity we refer to [?, ?]. 
Let uj, uj^'^, 2^ and 2^"^ denote the set of the Natural numbers, the set of 
finite sequences of Natural numbers, the binary strings of length iV, and 
the binary strings of finite length. Other definitions are analogue. Let 
e denote the empty sequence. Remark that there is a natural bijection 
between uj and 2^'^, defined by: 

e^0,0^ 1,1^2,00^3,01^4,... 

[uj] is the set of nested sequences of Natural numbers, with finite depth. 
Mathematically, it is the closure of u) under the mapping f{S) = S^'^. 
Remark that there is a computable bijection between co and [uj] , therefore 
most complexity and computability results in uj also hold in [u] . 

An interpreter $ is a partial computable function from 2^^ x [uj] 
[uj]. An interpreter is prefix- free if for any x, the set of all p where 
is defined, is prefix-free. Let $ be some fixed optimal universal 
prefix-free interpreter. 

For any x e 2^'^, l{x) denotes the length of x. For any x e a;^'^, l{x) 
corresponds to the length of some prefix-free encoding of x on a binary 
tape: 

l{x) 

^(^) = X^21ogXj. 

1=1 

For x,y & [uj], the Kolmogorov complexity K{x[y), is defined as: 

K{x\y) = min{/(p) : ^{p\y) 1= x}. 

The Kolmogorov complexity of elements in 2^'^ is defined by using the 
computable bijection mentioned in the beginning of this section. 
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For Z G [cijJjQjA G w", denotes Q\...Qi. The on-line decision com- 
plexity is defined by: 

K(Qi Ai; g„ ^ A„|Z) = min{Z(p) : < n\^[p\Q\ Z) [= A]}. 

This definition differs slightly with the definition of [?], with respect that 
^ G tu" is chosen, in stead of A G 2". Also a shorter notation [?] will be 
used: 

K{x\y]) = K(0 xi; ^ Xn), 

3 Main result and proof tactic 

Proposition 3.1. 

3c > 03°°x, y G cu<'" [K{x\y T) + K{y\x t+) - K{x, y) > c{l{x) + l{y))] . 

In [?] and repeated in [?, ?], it is proven that for any n there is an x G 2" 
such that: 

K{K{x)\x) logn — log log n. 

Let y be the binary expansion of K{x). From this and equation (jlj) it 
can be shown that 

K{x) + K{y\x) — K{x, y) logn — log log n. 

By inserting zeros at the right places in x, ?/, it can be shown that there 
exists infinity many x,y with l{x) = l{y)'- 

K{x\y T) + K{y\x T+) - K{x,y) > 0(log/(x)). 

This shows proposition 13. II for a logarithmic term in l{x). It seems nat- 
ural to think that such a result can be improved to a linear term, by 
concatenating such strings. This is what eventually will happen in the 
proof, at equation (fT2l) . However, to be able to add up these differences, 
conditional complexities must add up in some way to on-line decision 
complexity, in what extend this is possible is still an open problem. Hap- 
pily, Lemma 14.41 can circumvent this, if some extra information is avail- 
able. This information is stored in sequences u and v and is added to x 
and y. Adding this information requires, some more bounds to make the 
proof work: (|TOl) . (iTTl) . The proof below provides all technical details. 
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4 Proof 

First some definitions and lemmas are given. f{x) ^+ g{x) is short for 
f{x) ^ g{x) + 0(1), and f{x) =+ g{x) is sliort for f{x) = g{x) ± 0(1). 
For any a,b & [u], a — > b means that there is a fixed p G 2^'^ with 
Kp) ^ ^(1); such that $(p|a) |= b. Remark that if a — > b, then 
K{a) K{b). The shortest program witnessing K{a\b) is denoted by: 

a*[b] = min{p : $(p|6) 1= a}. 

a* is short for a*[e]. Remark that: 

a*[b],b^ia*[b]r[b],b. (1) 

Lemmas 14. H 14. 2[ and 14.31 provide observations, known within the com- 
munity, and stated here exphcitly for later reference. 

Lemma 4.1. For A E lo'^'^ , 

J2 KiAlA'-^) ^ K{A) - 0{n). 

Proof. For U,V E uj, prefix- free complexity satisfies additivity [?]: 

K{U,V\W) K{U\W) + K{V\U*[W]). (2) 

Since there is a computable bijection between u and [u], this result also 
applies to [u]. Let U,V E [oo], since [/*[Ty], — > U, 

K{V\U,W) ^+ K{V\U*[W],W). 

Inductive application of both equations above on A^ proves the lemma. 

□ 

Lemma 4.2. For a,b E u and c E u^'^ : 

K{a,b\c) =+ K{a,b,K{b\a*[c],c)\c). 

Proof. The proof below, shows the unconditioned version of the lemma, 
since the proof of the conditioned version is the same. In [?] and exercise 
3.3.7 in [?] it is stated that for every w E uj, and n ^ K{w): 

log|{j9G 2" : ^{p) 1= w}\ ^+ n- K{w,n), (3) 

and 

K{w,K{w)) K{w). (4) 
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Therefore, for c constant, there are an 0(1) number of programs that 
compute a, b and have length K{a, h) + c. Let S be the set of these 
programs. Remark that the elements of S can be enumerated given 
a, 6, -ftr(a, h) and therefore, for any p E S, using (jl]), we have: 

K{a, h) =+ K{a, b, K{a, b)) =+ K{p). (5) 

By equation ([2]), we have: 

K{a,b) =+ K{a) + K{b\a*). 

The programs a* and b* [a*], can be combined into a program p computing 
a, 6. This program p can be constructed such that p — > a*, b*[a*], and it 
has a length below K{a, b) + c, for c constant and large enough. Therefore 
pes, and since b*[a*] — > K{b\a*) = l{b*[a*]y. 

K{p) ^+ K{a,b,K{b\a*)). 

Combining with equation finishes the proof. □ 

Lemma 4.3. For b G 2<'^, a, c G [u]: 

K{a,b\c) ^+ K{a,b*[a,c]\c) -2\ogl{b). 

Proof. The unconditioned version of the lemma is proven, since the con- 
ditioned proof is essentially the same. It suffices to show that: 

K{b*[a]\a,b) ^+ 21og/(6). 

Again the proof of the unconditioned version of this equation is the same 
as the conditioned one: 

K{b*\b) ^+ 2 log/ (6). 

Given b and K{b) all programs of length K{b) that output b can be enu- 
merated. By equation ([3]), there are maximally a constant such programs, 
therefore: 

K{b*\b) =+ K{K{b)\b). 
Remark that by the prefix-free code 61O62O... 6/(6)1 we have: 

K{b) ^+ 2/(6). 

Using the natural bijection between u and 2^'^, this shows that for n e uj, 
K{n) ^+ 21ogn. 

K{K{b)\b) ^+ K{K{b)) ^+ 2log K{b) ^+ 21og/(6). 

□ 
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Let Z G [uj], A,Q & uj"' for some n, and N & uj. For i < n, let Tj = 
(Qil^i) and T = (Ti, ...,r„). 

K(T) = K{A\Q^,N) 
K{Ti\Z) = K{A,\A'-\Q'-\N,Z). 

For some fixed A^, and for all i ^ n,we define the sets Si and the numbers 
So{T) = 2^ 

S^{T) = S,^,n{p:^p\Q\N) i=A,} 
-1 if\S,{T)\=0 
[log |S'j(r)|] otherwise. 



A lower bound for K{T) is now proven. 
Lemma 4.4. 

K{T) ^ min{N,Y,K{nU_^)-0{n)}. 

i 

Proof. For each z, a semimeasure P can be constructed using A^~^ ,Li-i,N: 

P{z) = 2-^'-|{p e : $(j9|A*-\Q\iV) i= z}\. 

Remark that P defines a semimeasure and that P is enumerable. P{Ai) = 
0, for some i, implies that no program of length can solve task T*, thus 
K{T) > N. In this case the lemma is proven. Assume \Si\ ^ 1 and thus 
P{Ai) > 0. By applying the coding theorem [?] on P, it follows that: 

L,_i-L, ^ir(T,|L,_i)-0(l). 

Summing over i, gives: 

Lo - L„ ^ 5;]K(T,|L,_i) - (6) 

i 

Let p be a program of length K(T), solving task T. It possible to append 
2N-KiT)-oii) (different strings of length A^ - K{T) - 0(1) to in order 
to obtain elements from Sn- Therefore: 

Ln^^N-K{T). (7) 

Observe that Lq = N. Combining equations ([6]) and (I7j) proves the 
lemma. □ 
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Proof, of proposition [Ql Let u,x,y,v G u"' for some n. Let 

Z = N,0,0,0,Ui,Xi,yi,Vi, ...,Un, Xn,yn,Vn- 

Define: 

'^yv,i (l/j j'Vilz ) . 

For X = ux, yv, let -Dx.i = and for i ^ 2 let: 

Dx, = h_,{Tx)-L,{Tx). 

Remark that: 

J2Dx,,=N-L,iX). 

Equations (HH), (HU]), and ([HD are now derived. 
• Let: 

^ux,i—l- 



Vi = D„ 



At the end of the proof u, x,y,v, N will be constructed such that 
equation IHM holds, and therefore, ^ K{Tx) — 0{n) for X = 
ux, yv. Since 

z^' — ^ Ui — > Li_i{Tux) 

we have by lemma I4.4t 

K(Tx)^5^K(Tx,)-0(n). (8) 

i 

Choose: 

y, = K{T,,y[z'^^% (9) 
By Lemma [4. 2^ it follows that: 

Kiu^Xilz''') =+ K{u„Xi,K{xi\u*[z%z'')\z^'). 

By equation ([1]), we have that ■u*[2;^*], 2;^* < > Ui, 2;^*, and therefore: 

K{x,\u*[z%z^') =+ K{x,\u,,z'') 
= K{xi\z^'+^) 
= K{T^,i) 



Therefore: 

K{u„x„K{x,\u:[z^'],z'')\z'') =+ K{ui,Xi,K{T,,i)\z^'). 

Remark that l{xi) = m, and therefore K{Tx^i) 21ogm. By 
Lemma (14.31) . we have: 

K{ui,x„K{T,,i)\z'''') ^ K{u„Xi,K{T,,y[z^'^+^]\z^')-0{\og\ogm). 
By definition of ([9]), this shows that: 

K{ui,Xi\z^') ^ Kiui.Xi.yilz"^') - O(loglogm). (10) 

• From equations ([T]) and IQ, we have: 

m,z'^^'^y:[z'^^%z^^^'. 

Therefore, 

K(t;,|z^^+3) ^ K(^^|^^,^4.+2) 

=+ K(^,b*[^^^+2]^24.+2) 

=+K(2/„^.|z^^+^)-K(?y,|/^+2) (1^) 

• In [?,?,?] it is shown that for all 171,10 there is an x G 2™ such 
that 

ir(ir(x|ty)|x, It)) ^ logm — log log m — 0(1). 

Actually, the unconditioned version is shown, but this version has 
the same proof. Fix an m large enough and choose Xi G 2™ such 
that by equation ([9]): 

K{y,\z'^+') = K{K{T,,r[z''^']\z'^^') 
^+ K(K(r.,)|/^+2) 
= K(K(x,|^^^+^)|x„^^^+i) 
^+ logm — log logm. (12) 

First using Lemma 14.11 then applying subsequently equations ( ITUI) , (|TT|) , 
([I2D, and dHD gives: 

^ 5^ K{ui, Xi, y,\z'') + ^ K{v,\z^'^+^) + 0{n) 

i i 

^ J]i^(i/,,x,|240 + J2Kiy^,v,\z^'+^) -J^Kiyilz^'^') + 0(nloglog 

i i i 

^ K{Tu,) + K{Ty,) - O(nlogm). 
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Let (., .) be a computable bijective pairing function such that for all 
a,beu, l{{a,b)) ^ l{a) + l(b). Let: 

y'i = {vi^Vi). 

To finish the proof it suffices to show that 

lix') + ^ iV ^ 0{nm). (13) 

Remark that because Xi G 2*", l{xi) ^ 2m and because Ui = K{Tx^i), 
Km) 21ogm: 

K<) + Kyl) ^ i{ui) + i{xi) + Kyi) + K^) 

< KD~) + '2m + 2 log m + KD~)- 

Choose = 3mn. For X = ux, yv, J2i ^x,i ^ ^ + 1, and therefore 
KDx,i) ^ Snlogm. This shows that for m large enough: 

Kx') + Kl/) ^ 3mn = A^. (14) 

This shows equation ffT^ . □ 

Corollary 4.5. For some c > 0, for all but finitely many n, there exist 
a z e 2^" such that: 

K{0 Zi] Z2n-2 Z2n-l) + K{zi Z2] Z2n-\ ^ ^^n) " K{z) ^ CU. 

(15) 

Proof. Let x', be as constructed in the proof. Let x'^ and y'^ be binary 
prefix-free encodings corresponding to the definition of Kx). of and y^, 
i ^ n. Define z: 

z= x'x 1, 0, x'^^^j-^-j, 0, 

0, 0, y'i^i(^), 

0, 2/'„,l, 0, y'n,l{^)- 

Since ^j<„ /(x-) + /(y,') ^ 3mn, we have that z G 2^^"-. This shows that 
for all but finitely many n a string of length maximally Gmn exists that 
satisfies the inequality of the lemma. By appending zeros to the end of 
x' and y', equality ( |T5i) can be satisfied for every n. □ 
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